Drone interacting with a stranger having a cellphone

ABSTRACT

A pedestrian with a cellphone is in a public area. He sees a drone airborne nearby. He rents the drone to control its actions. His phone shows a signal encoding his electronic address, like a phone number. The drone decodes and sends a message with an URL, deep link or linket. The latter is a brand of the drone owner that maps to a deep link. The deep link designates an app in an app store. He installs the app and interacts with the owner, taking control of the drone for a specified time. The app shows images from the drone camera. The drone can crowdsource public safety. And check the presence of players at locations in Augmented Reality games. It can distribute electronic prizes to players. It can distribute keys for cryptosystems. The drone can pick up data from users at different places. Drone-drone interactions can optimise drone routes. Drones can be used with a blimp and electronic billboards to increase crowd use of an app.

REFERENCES CITED

“Apps everywhere but no unifying link” by C. Dougherty, New York Times,5 Jan. 2015.

“Deep linking's big untapped potential” by M. Thomson, VentureBeat.com,9 Aug. 2015.

“Generating and presenting deep links” by V. Tankovich et al, US PatentApplication 20130110815 (Oct. 28, 2011).

“Smart link system and method” by J. Chor, U.S. Pat. No. 8,433,800, (28Feb. 2011).

“Deep linking to mobile applications” by S. Saxena et al US PatentApplication 20150154644 (Dec. 2, 2013).

TECHNICAL FIELD

The invention describes a drone and its operator interacting with astranger having a cellphone.

BACKGROUND

Drones are now commonly used for civilian applications. They arecontinuing to improve in functionality. Regulations for how drones canbe used are in a state of flux. While at any given time the laws aboutdrones in a given country are fixed, this can be expected to vary overtime. And laws in different countries about drones can and does vary.

For recreational purposes, a drone is often controlled by a user (who isusually the owner) on the ground. The user might have a controllingdevice like a tablet or cellphone. Currently in the US, the user must beable to see the drone. And the drone cannot fly directly over peopleunless they are part of a group using the drone.

The most common group use of drones might be drone racing. A group ofusers races drones across an obstacle course, in the presence ofspectators.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a drone, its operator and a stranger with a phone.

FIG. 2 shows examples of the phone screen.

FIG. 3 shows a conversation between Jill and Bob.

FIG. 4 shows a deep link or linket in Bob's screen data.

FIG. 5 is a flow chart of how Bob can rent Jill's drone.

FIG. 6 shows text that the drone can send to Bob's phone.

FIG. 7 shows a player with a phone and a drone in an AR game.

FIG. 8 shows a game using a drone as a checker.

FIG. 8a is a flow chart of the checker.

FIG. 9 shows 2 drones delivering or picking up data.

FIG. 10 is a flow chart of a blimp and drone.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

What we claim as new and desire to secure by letters patent is set forthin the following claims.

In this submission, the term phone means a mobile phone. And it includesthe case of using the device known as a smartphone. More broadly, wewill often describe a person using a phone. This can be extended to theuse of other types of mobile electronic devices or wearable devices.

A drone is perhaps more formally known as an Unmanned Aerial Vehicle.

The submission has the following sections—

0] Motivation;

1] Basic scenario;

2] Interaction via a specified app;

3] Orientation of the phone;

4] Crowdsourced public safety;

5] Extensions;

6] Drone Rental;

7] Drone and Augmented Reality;

8] Checker;

9] Drone delivery of virtual items;

10] Distribute prizes or keys;

11] Drone pickup of virtual items;

12] Drone-drone interactions;

13] Blimp;

14] Billboard;

0] Motivation;

There is a user with a cellphone. Call him Bob. He is outside in apublicly accessible area. He looks up and sees a drone. A civiliandrone, and not a weaponized military drone. It has a camera (possiblyseveral cameras). He wonders one or more of several questions:

Who is operating it?

What is it doing? Yes, it's flying, but is it doing or going to be doingsomething more specific?

Can I communicate (ask) with the operator?

Can I see what it is seeing (through its camera)?

Can I control it? (aka. Can I rent it?)

1] Basic Scenario;

The situation in Section 0 is not the common one where Bob is theoperator of the drone. Or where Bob is a friend of the operator, and thetwo of them (perhaps with other friends) have gathered to see andcontrol the drone. In this submission, Bob does not know the operator,in general.

See FIG. 1. It shows Jill 100 as the operator of drone 101. She has acomputational device 102 with which she communicates with and controlsthe drone. Drone 101 has a camera 103. Device 102 can be a mobile phone,a laptop, a personal computer, a Heads Up Device (HUD), an AugmentedReality (AR) device or some other computational device. Device 102 mightalso be connected to the Internet, and thus to various servers. Theseare collectively indicated as Server 106.

The drone can also have Internet access. It has GPS or equivalentgeolocation ability. For simplicity, GPS satellites are not explicitlydesignated in FIG. 1. Jill's device can find the drone's location and,for example, place it on a map on the device screen.

The drone is flying. Jill might not be in line of sight or within somespecific distance from the drone. Currently in some regions, there maybe regulations requiring one or both. But regulations vary with regionand over time. For drones, the situation is in flux. Plus, there couldbe various categories of licenses. Jill can have a license as anexperienced operator, letting her do the actions described in thissubmission.

The arrow from 102 to the drone in FIG. 1 indicates that device 102controls the drone. This could be by 102 and the drone using standardradio methods. Or 102 could be accessing the Internet and likewise thedrone accesses the Internet. Then both devices might be contacting, say,an app server at server 106 and interacting via that device.

Bob 104 has a cellphone 105. It is assumed that the phone has a screenand wireless connection to an electronic network. We take this networkto be the Internet. While we say device 105 is a cellphone, it couldalso be other types of mobile electronic devices, like a tablet or aHUD.

Jill might be operating the drone in a purely recreational mode. Orperhaps the drone is patrolling an area. Akin to police choppers in anurban area. But the drone can complement the choppers and has someadvantages over them. First, a drone is much cheaper than a chopper. Afew thousand dollars versus several hundred thousand. Second, the costof Jill's time is much less than than of the chopper pilot. In partbecause of the physical hazard to the pilot, while Jill is safely on theground. The pilot needs more training. Third, the drone can fly at muchlower altitudes than a chopper. Due to its size, the chopper has to flyhigher to avoid power lines and buildings. To follow a suspect, thedrone can be much lower and closer, making it harder for the suspect toget away.

On patrol, the drone could be operated by one or more of the police,fire brigade or ambulance services. This could be a joint operation. Or,non-exclusive to those, the drone could also be operated by a citydepartment. Some cities have human guides who walk through the streets,to assist pedestrians. The drone might be a complement to those, just asit could be to the police choppers.

If the drone is deployed in these professional uses, it could bedecorated with insignia perhaps visible to Bob that say ‘police’ or‘guide’. This can answer the first 2 questions of Section 0. But if thedrone is operated by a private individual, the questions still stand.

Bob runs a program on his phone that shows a picture on the screen. Heraises his phone with the screen possibly pointed towards the drone. Viaits camera, the drone can have pattern recognition software that detectsBob's profile and attempts to decode the image on his screen. If itdetects Bob's profile, eg. a person with a raised arm, it can assumethat he is trying to contact to it via his phone screen. The drone cando one or more of these actions. It might get closer to him. It mightzoom its camera (if the camera has zoom capability) onto the phonescreen.

To aid the drone detecting Bob, his device might emit some combinationof alternating visuals. Some cellphones have apps that let the phone actas a torch. Likewise, Bob's device could emit light in some pattern akinto an SOS signal. Though this submission explicitly discourages theactual use of an SOS. The latter should only be for emergency purposes.But, for example, his device could emit visual Morse code (long durationfor a dash, shorter duration for a dot) by blinking the screen on andoff. The code could represent a short signal.

Bob might orient his screen towards the drone, and he holds up thedevice. This combination of his posture, with the upraised arm holding aphone, and the emission of a pulsed signal can let the drone more easilydetect Bob.

A variant is where Bob's device might also emit an audio signal.

The software run by the drone can be located in a combination of beingin the drone and in Jill's controlling device 102. Some of the actionsmight also be done or initiated manually by Jill. For example, when thedrone initially detects Bob's profile (say with his arm raised andholding what appears to be a phone), its software could flag this andalert Jill. She can decide whether the image suggests that Bob is tryingto contact her and whether she wants to respond affirmatively. If thelatter, then she might press ‘yes’ (or do whatever equivalent action ismade available to her via her device), and the drone takes the aboveautomated steps. Or, some of those steps might be manually done by her.In part, this could be when some steps have not be automated (yet), orbecause Jill overrides the automation for various reasons possiblyexternal to this submission.

What is the visual signal on Bob's phone screen? See FIG. 2. Item 21 issimply a short text, like his phone number or other electronic address.Item 22 shows a QR code. Item 23 shows a Data Matrix code. The barcodesencode the same data as item 21.

(If Bob's phone had earlier shown some Morse code-type visual signal onits screen, then as the drone detects Bob, the program on his phone canreplace that signal with other data in the format of the examples inFIG. 2.)

If the screen shows text, there is standard Optical CharacterRecognition (OCR) software that can decode it. If it is a barcodewritten in QR or Data Matrix or perhaps another barcode format, there isor can be expected to be standard decoding software available to thedrone.

Suppose Bob's screen showed his phone number, either as text or decodedfrom a barcode. The drone can have software to ring his number and talkto him. The drone can have an automated assistant, akin to Amazon Corp.Alexa, or Apple Corp.'s Siri. Note that the drone's assistant is likelynot to be on the drone. It runs on a server, which does not have to beand likely is not Jill's controlling device.

The assistant can then answer some or all of Bob's questions, usingspeech recognition to decipher his speech. For some questions, theassistant can contact Jill, who then manually talks to him.

If there is no assistant, the software on Jill's device can let herhandle all the interactions by talking to Bob.

Or the assistant or Jill could interact with Bob via sending him textmessages. He might respond by also sending text messages to them or byspeaking into his phone. The latter might be easier for him under somecircumstances.

FIG. 3 shows a possible interaction between Jill and Bob. This treatsthe input as either text or speech. Largely the submission treats theseas equivalent.

We have a new type of social interaction. Between a drone operator and astranger. This differs from a current common group situation usingdrones. A group of people gather together. One of them (“Jill”) operatesthe drone. The others stand around and watch the drone. If they alreadyin proximity, they can simply ask Jill without using their phones.

2] Interaction Via a Specified App;

The previous section had Bob giving his phone number on his screen, withthe subsequent interaction being via text or speech. Other ways aredescribed here.

The signal on his phone, in decoded form, could be the Internet addressof his phone, with an optional port number. See item 41. The Internetaddress is the (likely) Internet Protocol Version 4 temporary address ofhis phone. A similar example could be given in IPv6 notation. There isan implied app that Jill can use on her device, or an impliedApplication Programmer Interface (API). If the latter, then there couldbe several apps that Jill can pick, all using that API.

She then interacts with Bob. The point about using the app or API isthat the interaction is more than just the simple text or speech ofprevious section. Instead, there can now be a greater use of thegraphics of the screens of their devices, and possibly other hardwarefeatures of the devices.

How does Jill know what implicit app or API to use? One case is if fordrone interactions (or broader type of interactions), there has emergeda de facto app or API that is the market leader for such interactions.But suppose there is no such leader. Or, if there is but Bob wants touse a different interaction, requiring a different app by Jill. See item42. It shows a deep link. This can be regarded as a generalized URL. Ithas the format of an app id, followed by “://” and then Bob's IPaddress.

Note that currently there is no industry wide accepted format or indeeddefinition of a deep link. The format depicted in item 42 is meant to besuggestive by mimicking that of a conventional URL. Other formats arepossible.

For clarity, item 42 omits a port number. The app could have a hardwireddefault port. The “appId” string in item 42 symbolically designates anid of an application. This id could be with respect to an app store, formobile or fixed devices.

Which app store? If Jill is using a mobile device, there are currently 2major app stores, for Android and Apple. If Jill is using a PC, therecould be other app stores. Clearly, item 42 could for example have anextra field, perhaps as the first field, with an id of an app store.

A variant is item 43. The first field is an id of an API (ApplicationProgrammer Interface). There is assumed to be a registry of APIs. Thisregistry maps an API id to ids of apps in various app stores, where theapps all satisfy that API.

FIG. 4 has internal white spaces in the examples. There is no technicalsignificance to this. They are simply for visual clarity in the figure.

An alternative to items 42 and 43 is for the raw network address to bereplaced by a string. This string is mapped to an actual address at aserver. A device using 42 or 43 would submit the string to the serverand get the address. For simplicity this is omitted from FIG. 4.

Another variant is item 44. Bob shows or encodes on his screen hislinket [Bob Wang]. In our US patent application 20170109814, “Linket tocontrol mobile deep links”, and later applications, we described alinket. It is a personal brand owned by a user, similar to him owning aconventional domain. The linket is written in Unicode (ASCII is agrandfathered subset of Unicode). It can have internal whitespace andcan be case sensitive. The enclosing square brackets are arbitrarydelimiters. The example could have equivalently been written as >BobWang<, or {{Bob Wang}} using other symbols as delimiters.

A linket maps to a deep link, where the deep link can be in the formatsof items 42 and 43, for example. There is a linket Registry server onthe Internet. It takes as input a linket. It returns the currentcorresponding deep link.

Consider items 42 and 43. They show the current Internet address ofBob's phone. Often it is temporary. He typically connects to the

Internet via his phone carrier (e.g. in the US this could be ATT orVerizon) or via a nearby hotspot using WiFi or WiMax (e.g. in China).The problem is that the deep links of 42 and 43 are hard to remember.They have no intrinsic semantic value. And a second problem is that theyare temporary. If Bob moves to a different location and in the contextof this submission, he interacts with a different drone, he would have adifferent deep link in items 42 or 43.

In contrast, Bob's linket remains constant. Chosen by him so others canfind it easy to remember. FIG. 4 depicts his choice as simply being hisname. That is not required. He might have picked [Gamer Dude] forexample, if he was a frequent game player, so we see that his name doesnot need to be in the linket.

Given the choices in FIG. 4, suppose Jill installs an app to interactwith Bob. What can happen next?

Via his app, he might ask Jill for a video feed from the drone. He wantsto see images from the drone's camera as it proceeds later. For example,suppose the conversation in FIG. 3 happened inside apps used by them.Bob is interested in seeing what Jill sees when she sends her drone tothe rooftop. He might have been unaware of a party there. Or perhaps hewas aware, but he could not get an invitation. Jill can use her deviceto grant him a full or partial video feed. Perhaps a partial feed ifthere are bandwidth constraints. By partial feed, this could mean saygetting 3 frames a second instead of a full and higher frame rate.

She might require payment. She monetises her drone. In personalrecreational uses of drones, apart from drone racing, there arecurrently few ways to make money. One utility of the submission is todescribe such a method. The payment could be for a partial feed. If Bobwants a full feed, he pays more.

Suppose the drone also has a microphone; possibly directional. Thediscussion in the immediate previous paragraphs could have been confinedto the video feed (full or partial) being just images. Whereas for Bobto get audio might entail another payment.

The app could also let Bob get a flight path of the drone. Either in 2dimensions (in the horizontal plane with latitude and longitude) or in 3dimensions (including altitude).

Depending on the app, this path could be shown overlaid on a street planon his mobile device, if he is in an urban area.

Variants are possible. Suppose Jill operates the drone as part of herjob working for the city, as discussed earlier. The flight plan couldalso be or include the regular path taken by the drone while on patrol,in addition to the current real time actual path it will take.

Another possibility is for Jill to let Bob take partial control of thedrone. For payment. Another way to monetise. The attraction is to givepeople who do not currently have a drone a chance at operating one. Akinperhaps to go cart speedways at some amusement parks. The assumption isthat the drone has built-in collision avoidance firmware. And Jill wouldlikely maintain a manual supervision of the drone.

An analogy with current practices in some cities might be useful. In LosAngeles, for example, it is possible for a pedestrian to rent a bicyclewith his credit card. There a racks of special bikes scatteredthroughout the city. He can borrow and return a bike. In the presentsubmission, this corresponds to Bob outsourcing the ownership of a droneto third parties.

Whether or not Jill lets Bob might be predicated in whole or part byinformation about his part experiences (if any) with controlling drones.There might be data about this accessible to Jill's app.

Jill could also offer different regions accessible to Bob if he controlsthe drone. If he pays more, for example, he gets to control the droneover a larger area. The areas can be delimited and defined to the dronevia geofences. It is already a common idea for off-limit areas (asdefined by a government) to be built into a drone, so that its regularoperator cannot fly into those areas. This submission lets the operatordefine extra areas as a means perhaps of garnering more revenue.

3] Orientation of the Phone;

Consider when Bob initially puts an image on his phone and he holds itup for the drone to scan. In the daytime, sunlight can make the screenhard to read. The drone could have firmware to let it maneouvre itslocation vis-a-vis the sun's location and the phone screen's orientationto let it more easily scan the screen.

Consider the phone. It knows the time and its location (via GPS). It canaccess a database or use a formula giving it the sun's location in thesky (azimuth and elevation) at that time. Now suppose the phone has acompass, so that it can tell its azimuth. And perhaps a sensor (like anaccelerometer) so that it knows its elevation. The phone can find anoptimal orientation and indicate to Bob via sound or vibration to holdit up around that orientation. For example, for the azimuth, which isthe horizontal orientation, the phone can emit a varying pitch. Where inthe desired optimum azimuth, the pitch is a maximum, and as Bob moves itaway in either horizontal direction, the pitch falls. Likewise, forelevation, the phone might vibrate at a certain maximum frequency at thebest elevation, and then stop once Bob holds it there.

Or instead of vibrating for elevation, the phone varies its sound volumearound the optimal elevation.

4] Crowdsourced Public Safety;

Consider the case of Jill operating a drone for a city. One major use ispublic safety. Bob could be a witness to a hit and run, where a car hasdriven off. Perhaps he took a photo or a short video of it. In lawenforcement a vital idea is the lag time between an incident and when itis reported. Reducing this can greatly improve tracking a suspect.

One use of city drones could be to have them patrol streets where police(in cars or on foot) are not currently. This supplements the safetycoverage of a city.

Consider the photo or video that Bob took of a suspect or the suspect'svehicle. Why can't he just call the standard emergency number (911 inthe US) on his phone? First, calling the number does not let himtransmit the images or video. Second, there likely is no police near himcurrently. Time is of the essence. Suppose a drone approaches and hecontacts it via the methods described earlier. He and Jill interact viaan app that lets him upload the data. This can immediately betransmitted to law enforcement. Plus, the drone which perhaps now isunder the active control of Jill, can be used by her to look for thesuspect nearby.

A variant is where Jill is using the drone recreationally. Then via theapp, Bob uploads the images of the suspect to her. She of her ownvolition searches for the suspect. If she sees the suspect or vehicle,she follows via her drone. Law enforcement typically does not encouragecivilians to do so in person because of the chance of danger to them.But if she is not near the drone, there is little risk to her. It shouldbe stressed that this is not vigilantism. Jill as a private citizen doesnot actively apprehend the suspect.

A boon to law enforcement and a key utility of this submission.

This drone use also supplements surveillance cameras in public locationsand shops. Video data from those can be accessed by law enforcement butthis can typically take a day or longer. The data is rarely availablefor active (real time) pursuit.

5] Extensions;

Section 2 described Bob being able to get a data feed from the dronenear him. Or to partially control it. Suppose Jill controls a group ofdrones. If she interacts with him via a custom app, that app could lethim pick which drone to watch or control. A straightforward extension ofearlier section.

Thus far the interaction between Bob and Jill's drone has been initiatedby him. A variant is where she manually initiates, or her drone via itsautomated procedures initiates the interaction. The drone could approachBob. It could have insignia or logos or trademarks (we regard theseterms as synonymous in this submission) that tell Bob he can interactwith it. Similar to how taxis often have a well known brand or colour orlogo, and they cruise busy streets. Pedestrians can then hail them.

In the current case, when the drone approaches near Bob, this can inducehim to show it his phone screen.

6] Drone Rental;

In sections 1 and 2 and the associated figures, Bob was described aspassing to Jill a means of Jill (the drone operator) contacting him. Bobpicks an app and he might have a label or brand, called a linket, viawhich Jill accesses. The current section describes an alternatescenario, where it is Jill who has a brand and Bob decides whether tointeract. One motivation for him can be to rent the drone. The rentalhere does not require him to physically hold the drone. The drone canremain airborne and outside his reach. For the drone owner, this reducesthe chance of a rogue Bob taking her drone.

See the flow chart in FIG. 5. Item 501 is Bob showing an electronicaddress on his cellphone screen. As earlier, this can be his phonenumber, for example. The address can be shown as plain text, or encodedas a barcode. Item 502 is the drone decoding it, using OCR (for plaintext) or an appropriate barcode decoding method. When item 502 says thedrone does this decoding, an alternate method is for Jill's controllerdevice in FIG. 1 to do the decoding. If the drone decodes, it then sendsthe decoded address to Jill's controller.

Item 503 has Jill's controller sending an electronic message to Bob'sdecoded address. Suppose the latter is his phone number. The message canbe sent as SMS or MMS. For SMS there might be a maximum character limit(perhaps 140 characters). For MMS the limit would be greater. Note thatover time, the limits could increase or that SMS and MMS might becomeequivalent.

Or suppose item 503 has Jill sending the message to Bob's email address.In either case, Bob's phone gets the message and displays it. Item 504asks whether the message is an URL. If so, we go to item 505, where hisphone loads the page pointed to by the URL. An example of such an URLmight be item 61 in FIG. 6. The page could show information about Jill'sdrone as well as other drones. The latter might be owned by otherindependent operators.

If the page shows several drones, then which is Jill's drone, which isnear him? He does not want to control another drone at another location,presumably. Hence the URL 61 might in practice be a longer URL, pointingto a page just about Jill's drone.

If Bob finds the terms in the page agreeable, he could rent the drone initem 509. The financial mechanism is left outside the scope of thissubmission. It could involve traditional credit card or debit cardmeans, or Paypal or a synthetic currency like Ethereum or Bitcoin.

In item 504, suppose the message is not an URL. We go to item 506. Isthe message a deep link? If so, go to item 507. An example of a deeplink is item 62 in FIG. 6. The droneRentalApp is a symbolic placeholderof an app id in a mobile app store that Bob's phone can use to installthat specific app. The 23.120.1.58 is the temporary IPv4 address ofJill's drone or controller. Hence Bob's device can install this app ifit is not already on the device, and then the app is run and contacts aprogram on Jill's controller in item 508. Bob uses the app toinvestigate the rental terms. These can include more information aboutthe capabilities of the drone, like the number of cameras, theresolution, the color depth, the number of frames per second etc. If allis ok, he goes to item 509 to do the rental.

But in item 506, suppose Jill's message is not a deep link. Go to item510. Is it a linket? If so, go to item 511. An example of a linket isitem 63 in FIG. 6. Jill has a brand expressed as the linket. She ownsthe linket. Bob picks it in item 511. Leading to his device sending thelinket to a linket Registry and getting back a deep link like item 62.We go to steps 507 and 508.

Thus far the flow chart has 3 key decision points, 504, 506, 510 thatparsed Jill's message. Other types of structured data in her message arepossible. These are implicitly given by the arrow for ‘n’ (“no”) in item510.

Bob's motivation in this section is not restricted to renting the drone.He might simply be curious about one or both of the drone and itsoperator.

The business in this section predicts a future where drone use andownership are common; likely more common than now. But deliberately, usedoes not have to equate to ownership. Bob might actually have his owndrone. But it is not with him when he sees Jill's drone. Plus theremight be a specialisation in drone models. (Like for motor vehicles.)There could be common, cheap drones, and Bob owns one of those. But Jillhas a more expensive drone, with perhaps longer flying time, or a higherresolution camera (or cameras). Or suppose that there are governmentregulations on drone usage, with average users like Bob being subject tomore restrictions. Jill could have taken more extensive drone lessonsand have a license that lets her rent out her drone, with perhaps herhaving a supervisory overriding role for safety. Bob might be able touse her drone under conditions that he cannot with his drone.

This section described how the drone controller, Jill, could show alinket of hers in the message sent to Bob. A variant is where the linketis a brand owned by a different entity. That entity might want Bob todownload and use a given app. Its linket maps to that app. It hires Jillto fly her drone as advertising.

7] Drone and Augmented Reality;

Augmented Reality (AR) games have been popular; notably Ingress andPokemon Go from Niantic Inc. Both have the property of associatingentities (“portals” in Ingress and “monsters” in Pokemon) with reallocations. Players go to those places and interact (e.g. “capture”) withthe entities and each other.

Now imagine a drone as an integral part of an AR game. Jill might be aplayer or she might be hired by the game company. In either case, sheoperates a drone as in FIG. 1. Bob is a player on the ground, and ingeneral Bob and Jill do not know each other. Bob uses his phone to playthe game. See FIG. 7. It shows Bob 70 with his phone (or other mobile orwearable electronic device) 71. A game runs on 71. It talks to itsserver 75.

One possibility is that when he sees a drone, he runs the game andpoints the phone camera at the drone. The game analyses the photo or avideo of the drone's movement. When we say ‘the game analyses’, this canbe done by the game app on Bob's device, or by the game server, wherethe game uploads the photo or video to its server. Hence, when we saythe game does some analysis or action, this might be done at the levelof the game app or at the game server.

The game has pattern recognition to deduce if the drone is in the gameor not. Or the drone has visible insignia that Bob sees and tells himthat it is in the game. If so, he might do some action in the game totell it that he has manually detected a game drone.

In either case, the game could show some image associated with or nearthe present location of the drone. He might now have to “capture” or“shoot” the image. The image could represent a character in the game.

Why cannot the game just send the coordinates of his mobile device tothe game server, which knows if and where any drones associated with thegame are? This is possible, but the above assumes that the game requiresBob to do certain manual steps.

The difference with the earlier cited games is that those usedcharacters tied to fixed locations. Now a character or entity (e.g. arefueling station for a player like Bob) can move. Jill's task could beto control the entity via flying the drone, where the players chase theentity or drone.

One scenario is that as the drone moves, Bob has to follow, so his gamecan continue to detect the drone and the entity associated with thedrone location. Note that the entity does not have to overlap the dronelocation in the game. The entity could be displaced or offset from thedrone. Specifically, the entity could be depicted as moving on theground, under the drone.

Another scenario is that the entity does not move as the drone moves.Bob just had to ‘capture’ the entity based on a temporary location ofthe drone.

Suppose there are several drones in the game, each associated with adifferent entity. How does the game know which entity to display when adrone is seen? One answer is if the game does pattern recognition on thedrone. Perhaps the different drones have different colours. So a reddrone maps to a refueling station, while a blue drone maps to an ammodump. Etc.

Thus far in this section, Bob has not directly connected to Jill. Oneway is for him to show an image on his phone screen, as in earliersections. This has the advantage that it can tell Jill that Bob isplaying. Otherwise unless Bob is (say) wearing a game costume (perhapsas cosplay), Jill cannot a priori tell Bob from a non-player.

Another scenario involves Jill playing the game. The game might bedesigned to let players use a drone as an extra mobile device, or thegame might be restricted to only players using drones (along with theirphones).

Consider Bob playing a popular current game like Pokemon Go. Bob playsusing his phone, with its camera and GPS. The game on his phone knowsits location via GPS. The game uploads the location to the server. Theserver sends information about any nearby monsters to the game. The gameshows these on the phone screen, overlaid on a street map. For example,this could be Tiger 72. Server 75 associates this with area 73, alsolabelled as Alpha in FIG. 7. Tiger 72 wanders in Alpha.

Return to Bob. He controls his drone 76 via a mobile electronic devicehe is carrying. The drone has built in GPS by assumption. (In 2017 thisassumption is de facto true.) The drone is also assumed to have a camera77. The game app on his mobile device has been modified so that it takesits location not from his device GPS coordinates, but from thecoordinates of the drone. Note that currently, when Bob controls thedrone via his device, the drone sends its location regularly to hisdevice. So the latter can, at the very least, show the drone's locationon a map on Bob's device. It is a small step to then modify the game appto use the drone location.

Likewise some current AR games also might use a compass or gyroscope inthe phone to deduce the direction that the phone is pointing. Where thismight be defined as the direction of, say, the camera on the side of thephone opposite to the screen. The drone is likely to also know thedirection that its camera is pointing. It is a small step to modify thegame app to use the drone camera's orientation, instead of theorientation of the device 71 that the app runs on.

Hence Bob plays the game using as input the drone's location andorientation. (The latter being defined as the orientation of the drone'scamera.)

We described how the game might be altered to allow this. A differentmethod is for Bob's mobile device to have a program that acts as anintermediary. The game sends a query to the phone operating system forthe phone's location and orientation. This query is received by theprogram, which replies with the drone's location and orientation. Thisprogram can perhaps be regarded as a ‘container’ or framework in whichthe game is played.

A game could be designed or altered specifically for different and newtypes of gameplay when drones are used. For example, a game could havemonsters or portals at locations not easily accessible to players atground level. A location could be a rooftop of a building 74. In a ruralarea, a location could be a flat area on the side of a mountain, whereit is unsafe for players to climb to. Hence using drones can expand thetypes and locations of gameplay.

For example, in FIG. 7 Bob sees Tiger when he gets near location Alpha.A task in the game is for him to neutralise Tiger by moving it to alocation where it cannot get at the players. By controlling drone 76, hedirects it to hover near location Phi on the roof of building 74. Heorients drone camera 77 to point at Phi and he can see images of Phi onhis phone. Phi is vacant. The game lets him capture Tiger and move it toPhi.

There is another aspect here. The drone has a camera and GPS andorientation data. The GPS location is the location of the drone. Notnecessarily that of Phi on the rooftop. But this is similar to thecurrent situation of the phone's location. It is not necessarily that ofAlpha. But just as current AR games can compensate for thismathematically, so too can the same methods be used for the drone.Essentially, given knowledge of the drone location and the direction ofits camera, and some image recognition, a nearby area Phi can bedetected by the game. Put plainly, knowing the drone location and adirection vector (the orientation of its camera), an actual locationused by the game can be that drone location plus (meaning a vectoraddition in 3 dimensions) a displacement.

What is this displacement? The drone has the (x,y,z) of its location. Ithas an angle Theta of its camera orientation. The image from the camerashows a rooftop, say. But how far away is it? There are several answers.One is that the drone has a rangefinder, like a laser or sonar. Anotheris that various heuristics (rules of thumb) can be encoded in the gameto give a plausible estimate of the distance. Or the game can useexternal databases, like Google Maps. These databases can have maps withbuilding locations and elevations. In some cases, this can be used tofind the displacement.

There can be many variations to using the drone in a game. Bob mightneed to have something visible on his phone camera and and a differententity or location visible on the drone camera. When both conditions aretrue, he can do some action not otherwise possible. Like swap theentities. Or the entities remain at their locations. But the game letsBob move some property from one entity to the other. The property couldbe gold coins, or a magic potion or a magic scroll.

Or the drone is used to bring characters into the game. There is noTiger at Alpha in FIG. 7. But Tiger is at Phi. Bob uses the drone tosearch rooftops until he finds Tiger on the drone camera, shown as an ARoverlay. He captures Tiger and releases it at Alpha. Now Tiger canperhaps be an obstacle to other players.

Or instead of Bob seeing Tiger in FIG. 7, he uses the drone to find asuitable rooftop Phi. Perhaps suitable is defined as him searchingvarious high places until the game announces that he found Phi (Phi'slocation was pre-defined in the game). Simply finding Phi could give Bobsome points in the game.

A variant is where a character might be initially visible on the groundto Bob using his cellphone. Then when Bob gets near it, it moves. Butcaution is required by the game designer, if the character moves atground level in an urban environment. Suppose the character moves on thesidewalk but Bob sees it on his phone when he is across the street. Ifhe is careless, he might chase it by running across the street withoutlooking for traffic. Now in this submission, the character might bedisplayed inside a park, say, where it only appears on Bob's phone whenhe is also in the park and does not need to cross a road. Plus when thecharacter moves, it climbs up some rocks or trees, so that it disappearsfrom his phone screen. He must use a drone to chase it. Its image willonly appear overlaid on the drone camera's screen. Safer for Bob.

By using a drone, the game can be extended to involving places notaccessible to players in a public area. Qualitatively different from twoplayers interacting in a public area in a game. Other examples ofdrone-only accessible areas could be the tops of tall trees, or privatewalled gardens.

While this section describes AR games, non-AR games can also be extendedto use drones in the above ways. For decades there have been scavengerhunts. Clues were physically scattered around public areas, like thestreets of a town. Players did not use phones. They had sheets of paperwith instructions to find the clues. One clue might lead to another.These games were extended to geocaching games once mobile phones withgeolocation became common. Geocaching let clues use lat-long locations.Clues did not have to be physically put at a location. Instead, in ageocaching game, when a player reached a given location as indicated byhis phone, the game on the phone could emit text that was the clue. Butin both cases players were restricted to areas accessible to the public.

A game running on Bob's cellphone could use exclusively a phone. Or itcould let Bob alternate between sometimes just using the phone (on whichthe game front end runs) to play, and sometimes also using the drone inthe manner described above. This can be when he rents the drone (perhapsbut not restricted to the manner described in earlier sections) from itsowner. The game can have a mode where he is limited to a maximumduration where he controls a drone. Or, aside from whatever he pays thedrone owner to borrow the drone, when he uses it in the game, he mighthave to spend a game currency in the game. Maybe because a drone giveshim extra powers in the game and thus in the game logic he has to expendresources to use it.

The game could allow different scoring for locations accessible only bydrone versus locations accessible on the ground using a cellphone.Perhaps the drone locations are worth more points. Or a game mightrequire that the players access a minimum number of drone locations anda minimum number of ground locations.

In more general terms, the above can be restated. A game runs on a gameserver and on corresponding game apps on mobile devices (usually phones)carried or worn by players. The devices have GPS and cameras. A playercontrols a drone. The drone has a camera and GPS. The drone cameraimages are shown on the phone. The server holds in memory characters inthe game. The game app might also hold a subset of this data. Associatedwith each character are various attributes, including a location. Thelocation can be given in 2 variables, like latitude and longitude. Itcould also have a third variable, altitude. A player uses the dronelocation to alter attributes of a character in the server. This includesin some cases the destruction or creation of a character. The dronegives players visual access to locations not otherwise accessible to thephysical presence of players.

Another method is for the drone to be a mobile hotspot. It is now commonfor a cellphone to be able to be a mobile hotspot, letting nearbydevices communicate with it via WiFi and thence to access the Internet.

The drone can spread the use of a mobile app, especially oneintrinsically for outdoors use. In the 2 main app stores, Apple andAndroid, there are currently about 2 million apps in each. A majorproblem each app faces is simply to stand out amongst its peers. The appfirm can hire a drone to fly around open areas having people. By themethods of this submission the intent is for users to interact with thedrone and install and run the app. The drone is used for both publicityand distribution. It acts as a new mechanism for the app to garnerawareness and use.

As part of the publicity, if a user Bob were to interact with the droneand download the app, he might get bonus points compared to someone whojust goes to the app store to find the app.

8] Checker;

Another use of a drone is as a checker. Consider a game where theplayers are outside, perhaps searching for clues or where there mightaugmented reality effects shown on their mobile devices. The gamerequires that the players actually be at specific locations. The gameexists on a game server 75 in FIG. 8. The players have the front end,the game app, on their mobile devices. See for example Bob 70 and hismobile device 71 in FIG. 8. But how to enforce the players being atthose locations? If the game has a monetary prize there could beincentive for players to cheat. Or even if there is no such prize, thechance of some recognition or even fame could induce cheating.

The basic problem is this. The game app is on a player's device. The apptypically gets the location from GPS. Perhaps in addition to also usingsignals from cellphone towers (aka basestations). The player controlshis device. A rogue player could install the app and do an unauthorizedmod (“jailbreak”) of the app. Or the app is run in an emulator orsimulator or software framework where false position information is fedto the app. The problem is fundamental. The game server is communicatingwirelessly with what it believes to be valid game apps sending it trueinformation. In a twist on a Turing Test, the server cannot easily if atall tell that the apps are behaving truly.

This section offers a solution. The server hires drone 76 (with itscamera 77) from a drone operator. This hiring could be done with somemanual steps by a person at the server or perhaps all the hiring stepscould be automated at the server. Or equivalently the drone operatorworks for the company running the server.

The drone is chosen to be close to a given (x, y) location where theplayers are meant to be or say they are. Thus if there are variousdrones at different locations, there might be some software that showsthese, letting the server (or its human operator) pick an availabledrone near (x, y) that approaches the immediate area around (x, y).

The server could have a programmatic control of the drone, where theserver gets a video feed (or some static images) from the drone camera.FIG. 8 omits for simplicity the possible presence of a human at theserver or a human operator doing manual control actions for the drone.

The game server is told by Bob's app that he (or more precisely hisdevice 71) is at or near (x, y), where “near” is defined by the game.One method of checking is for the server to tell Bob (via his app) toraise his hand or do some equivalent visually distinctive action. Thedrone has image recognition methods to detect Bob. Or the images couldbe passed onto the game server, which has these methods, or which has ahuman manually looking at the image feed.

A stricter test is shown in FIG. 8. The server sends a pseudo-randomstring (perhaps called a “nonce”) in step [1] to Bob's device 71. Theapp tells Bob to point the device screen towards the sky. This mightalso involve telling Bob to orient the screen towards a nearby drone.The latter could presumably be the checker drone 76. The app canautomatically show some representation of the string on the screen, inthe ways described in earlier sections. So the string might appear aslarge text or as a barcode.

The drone may get closer to improve its pattern recogition. It usesappropriate decoding methods to take an image of the screen and decodeto extract the string, as shown in step [2] of FIG. 8. The droneforwards the string to the server in step [3]. Preferably the drone canalso forward images of Bob. Thus the server can check that Bob is indeedat or near (x, y).

Suppose there are several people playing the game and they presumablyare near (x, y), as told by their devices to the server. The servercould ask some or all of these to do the actions of Bob. If some, itcould pick a random subset of the players. In general, the nonce sent toeach such player can be different. Note if there are several playersnearby, they need not be in a team. They might be present asindividuals. And of course around (x, y) there could be people who arenot playing the game.

It might not be necessary for the server to check all players near alocation. By telling the players beforehand that it will randomly pick asubset to ask can act as a deterrent. A rogue player will risk beingdetected as not being present to be imaged by the drone, when his gameapp is telling the server that he is present. Compliance to the game'srequest to the players would be a requirement of playing the game.

FIG. 8a is a flow chart of the drone being used as a check. Item 8 a 1is the player going to some location (x,y). This may be some speciallocation in the game on his phone. Item 8 a 2 is the game sending thelocation to the game server. Item 8 a 3 is the game server sending thedrone to (x,y) to check up on the player. Item 8 a 4 is the servertelling the game to show some unique image on the player's phone screen.This may or may not involve using the nonce. In either case item 8 a 5is the player showing the phone screen to the drone. Item 8 a 6 is thedrone decoding the image. Item 8 a 7 is the drone sending the decodeddata to the game server. Item 8 a 8 is the server confirming (or not)that the player has been visually verified to be at (x,y).

The method of this section also has a benefit of giving a revenue sourceto drone operators. Which increases the probability that such operatorswill be near places of interest to the game.

Another benefit of this method is to reduce a different cheating modewhere Bob has 2 mobile devices, each with a copy of the game app. Heruns one device as one player and the other device as a second player.The game might prohibit this. Perhaps Bob does this to try to win 2prizes. Bob runs the risk of being detected if both “players” are askedby the game to show nonces to the drone. The capturing of Bob's imagecan reveal this.

It can also work across different locations. At one (x, y), one of Bob'splayers was asked to show the nonce. Bob did so, and the server gets animage of Bob. Later at a different location, his other (fake) player israndomly chosen by the server and asked to show a (different) nonce tothe drone. His real image is taken and sent to the server. The servercan have image recognition that recognises that the images taken ofostensibly different players at different locations are in fact the sameperson.

The method is different from and has advantages over a human alternativewhere the game has a human checking players near a location. The latterhas practical difficulties. The game has to find non-player humans to becheckers. Consider such a human checker, Sue. If the area around (x, y)is large, she has to go near players or they have to come to her. Thistakes time and impacts the game experience for the real players. Whereasa drone can travel faster than Sue across the area and can check moreplayers in the same amount of time.

Plus Sue has to be trained in the specifics of the game checking steps.Whereas for using the drone, much or all can be automated. Even if thedrone operator manually supervises the drone, she in general does notneed to know the specifics of the checking steps.

Another use of the drone by the game company is to check the presence(or absence) of physical entities altered or placed by players. Supposethe game involves players putting a special flag in a location inside apark. If a player does so, the game tells her to take a photo of it atthe location. Perhaps with her or other players in the photo. Andpossibly the photo might have the location coordinates and timestampwritten into it by her phone. But a rogue player can modify the imageand metadata. A drone could fly by the location later and take a photoof the flag if it is there, as a check.

9] Drone Delivery of Virtual Items;

There has been much work in recent years on the use of drones to deliverphysical goods to customers. Amazon and Google, for example, have madetests and filed patent applications on this subject. In contrast, thissection of the submission argues that drone delivery of virtual itemshas several advantages. The virtual items are bit strings that aredelivered to mobile devices of customers, typically in open areas.

First, there is less total mass of the drone plus the item. The latteras a bit string is essentially massless. This means that a given dronehas a longer range and flight time. Or a smaller, cheaper drone could beused to deliver bit strings.

Second, it is safer. In case of collision with another object, orcompared with a drone dropping a physical item by parachute.

Third, the drone is more likely to make multiple deliveries. Meaning itcan deliver data to several customers on the ground. Consider what itmeans when the drone has to deliver a physical item. In most cases ithas to return to base to pick up another item. The return trip means anempty drone. Granted, because it is lighter, the trip can take lesstime. But still, approximately, the drone has a duty cycle of 50%.

Fourth, there is less chance of the drone being stolen. Compared to whena drone lands on the ground to deliver a physical item. The dronedelivering data wirelessly to a customer can stay in the air and out ofreach of people.

Fifth, there is less chance of the delivered item being stolen. When aphysical item is left at a customer location, if the customer is notpresent, the item could be stolen.

This section has argued ‘why’ for delivery of bit strings. The nextsection describes ‘how’ and offers 2 use cases.

10] Distribute Prizes or Keys;

This section closely follows the scenario of section 8. The drone can beused at an outdoor location to distribute prizes or points in a game toplayers nearby. The server has the app tell the players that the dronethey see nearby is run by the game. They are also told that if they, forexample, do something in the app that then displays a unique image onthe screen, and they hold up the screen to the drone, then it will bescanned. The image can decode to a unique identifier of the playerholding the phone. This id might be the player's name (or handle) in thegame. Or a temporary unique id assigned to the player for just thistask.

The game then awards some prize or points in the game to those playersscanned by the drone. It can be seen that while the motivation differsfrom section 8, operationally the steps are similar. The current sectionis akin to a carrot approach, while section 8 is a stick approach.

In related ways, suppose the app and its server is not restricted to agame. For a general encrypted interaction between users, or betweenusers and a server, a common problem is how to disseminate a key to acryptosystem to the users. The cryptosystem will do the encryption anddecryption. A drone can be used to spread the keys. See FIG. 1. Jill isthe operator of the drone. At some earlier time she physically has thedrone near her, on the ground. She can put keys into the drone inseveral ways. Each key is a bit string. The keys might be on a memorychip which she inserts into a slot on the drone. The keys can be read bythe drone operating system when it is airborne. Or Jill plugs a cablefrom her controlling computer 102 to the drone, or 102 makes a wirelessconnection. In the latter case it can be safely assumed thateavesdropping is unlikely. She and the drone might be in a closedshielded room for example. And the wireless transmission might be at lowpower because device 102 and the drone are next to each other.

A given key could be for user Bob. Bob uses the app to tell the serverwhere he is. The drone is nearby and is told by the server that a usernearby is that user Bob. The app tells Bob to raise his device screen,where the app is showing some image encoding some id of Bob. The dronegets near, decodes the image and checks that the id is that of Bob. Thedrone transmits by various means the key. By complementary means the appon Bob's device gets the device to receive the key.

One method is for the drone to modulate a visible light signal. Thedrone can have a light that can be used for this. Given that the totalsize of the key is small, a relatively low bandwidth modulation willsuffice. The light could have other unrelated uses. Bob's device (oftena cellphone) would use its camera to record and decode.

Other parts of the electromagnetic spectrum might be used. Perhaps for aBluetooth transmission. Or a WiFi transmission.

In this way, the key never travels on the electronic network (typicallythe Internet) between the server and Bob. Note importantly that Bob isnever required to have earlier met in person anyone from the companyrunning the server and (temporarily) controlling the drone.

But what if the transmission is detected by others near Bob? First, onecase is where Bob is by himself. If others are near, Bob might have theoption to tell the app that the drone should not transmit until he iselsewhere. Or the app can let Bob define a seed for a simple encryptionof the key by the drone. Or the app can define the seed itself. Giventhat the key is likely to be a pseudo-random bit sequence, thenencrypting it in the latter simple way can most likely throw off nearbyeavesdroppers. When the drone transmits the encrypted key to Bob'sdevice, his app can decode using the known seed and extract the key.

The main idea is that the drone acts as an out of channel mechanism forcrucial data to be transmitted to a user.

11] Drone Pickup of Virtual Items;

If a drone can deliver (virtual) items, it can also pick up (virtual)items. This section is a corollary to the previous section. It isanother difference from drone delivery of real items. For the latterthere is little discussion of a corresponding use of a drone to pick upa physical item. Perhaps in part due to practical difficulties of how adrone might be outfitted to carry a relatively unknown item supplied bya stranger. These could be a chance of the drone being stolen or damagedif it lands. Or the item might be attached wrongly and then drop fromthe drone when airborne.

For the pickup of a virtual item (i.e. data) none of these problemsoccur. The drone can remain airborne near the person, Bob, from which itis picking up. One case can be for gaming, and specifically for ascavenger hunt. Bob is outdoors and does certain actions in the game.After this, his game app makes some data acccording to various inputs.One input could be his location. Another input could be the time. Forexample, he might need to do the tasks in minimum time; perhaps racingagainst other players to finish the tasks first. His game app makes thedata. By assumption, the app is in wireless contact with the gameserver. The app could just transmit the data to the server to mark thatBob has done the tasks.

Instead, the game requires Bob to “send” the data to a drone near him.He might have to wait for the drone; perhaps it is busy servicing otherplayers at this or other locations.

When the drone nears, he could do steps like described in earliersections. He might raise his phone, screen facing away from him, andshowing some visual signal. There are at least 2 possibilities. One isthat the the drone images Bob, but the data is sent from his phone tothe server. The drone never gets the data. Here, as in earlier sections,the drone is used to get a photo of Bob at a place. This could be a keycondition of the game.

In a second case, the data is transmitted wirelessly from Bob's deviceto the drone. This could be via an image or video on his device screen.The data could be encoded as a barcode or sequence (“video”) orbarcodes. The data could be played as audio. Etc. In this case, the gamemight include a nonce (random bit string) sent from the server to hisdevice. The nonce would be put into the signal sent to the drone. Thistries to ensure that if say there are several people near Bob, that thedrone is getting data from the real Bob.

While the second case clearly has the drone picking up data, the firstcase does not. But in the terminology of the submission, we consider thefirst case to also be a drone “pickup”. The drone acts as the enabler,in the eyes of the end user.

12] Drone-Drone Interactions;

See FIG. 9. There is Drone 91 and Drone 92. Each has a different humanoperator. Or one or both might be entirely machined operated. In eithercase, they might be owned and operated by different legal entities.Following the previous sections, the drones are tasked with going todifferent locations and checking the presence of users of some apps(game or otherwise). They might be delivering data (keys tocryptosystems or game prizes). Or they could be picking up data. Drone91 might be delivering data while drone 92 could be picking up data. Orboth drones might, perhaps coincidentially be delivering data. Or bothcould be picking up data.

In the general area there are 5 users at different locations, Sue,Laura, Tim, Mike, Dave. Drone 91 is tasked to go to where Laura and Daveare. For visual clarity FIG. 9 shows “Drone 91” and “Laura” and “Dave”in italics. Likewise drone 92 is tasked to go to where Sue, Tim and Mikeare. These labels are shown in bold to aid understanding. Neither dronenecessarily knows any names of these users. Rather, each drone could begiven the coordinates of those users. The dashed lines for 91 show apossible trajectory for it to service Laura and Dave.

FIG. 9 can be for the case where the users are calling for drone pickup.Each user uses her app to call its server to get a drone. Or for thecase where a drone is perhaps to check for the physical presence ofusers at a given location, then imagine Sue being replaced by “Southwest corner of Central Park”. In this case the drone, the drone serverand the app server might not know whether any players are at thatlocation or not.

Laura and Dave might be using the same app or different apps. LikewiseSue, Tim and Mike could be using the same app, or 2 different apps or 3different apps.

Consider when the drones get their orders from the same drone schedulingcompany. The latter could be akin to a ridesharing firm like Lyft orUber. And the drone operators (assuming both to be human) might be akinto human drivers of cars who sign up with the firm. The company canreassign workloads to optimise (in this case minimise) the distancestravelled by each drone. For example, drone 91 now has to go to thelocations of Laura and Sue. While drone 92 goes to Tim, Mike and Dave'slocations.

The drone server differs from the app servers. The latter might contractwith the drone server to do various location-based actions like thosediscussed above.

Now suppose the drones are with different drone server firms. Routeoptimisation is still possible. One way is for the firms to exchangeroute information between their servers. A second method is for a droneto have pattern recognition of other drones. If drone 91 comes closelyenough to drone 92, 91 could look for a drone serial number or logo on92. There might be a requirement, like for ships and planes, for dronesto have such information in a standard format on the fuselage. 91 canlook up data on 92. The data could include an electronic address and ameans (e.g. by some industry API) for 91 and 92 to exchange routes andnegotiate to swap destinations in a mutually beneficial way.

This negotiation could be done manually by the drone operators. Orpartly or entirely computationally.

The above discussed a relatively simple optimisation of distancetravelled for each drone. Implicit was the expectation that the time ininteracting at each location is (roughly) the same. This does not haveto be true. The interaction time might differ for different apps. Hencethe optimisation could be of the time needed for each drone, where thetime is both the travel time and interaction time.

A further factor is the remaining flight time for each drone. Suppose 91is short on fuel. It can only go to Sue's location. Then its route isaltered to just Sue, while 92 visits all the other locations. This ofcourse assumes that 92 has enough fuel to do so.

A key property of this section is that the use of drones to carry or getvirtual data is essential in making the steps feasible.

13] Blimp;

In the US civilian blimps are often used at major sporting events. Theblimps hover over large crowds. Typically a blimp has a corporate logoand possibly an electronic billboard on which is shown advertising.

A blimp can be used with drones. The blimp might be a platform fromwhich drones are launched and to which they return, perhaps to rechargeor refuel before flying again. Or the drones might come from normalground based sites. These are not mutually exclusive. A blimp could havedrones from both origins.

A property of the earlier uses of drones in this submission is that adrone lacks a screen on which messages can be shown to users on theground. But if a blimp has a screen, it can show messages pertaining tohow to use or interact with the drones. For example, the screen mightshow a website URL (presumably short) or enough of a description so auser could use her phone to go to the webpage or go to the app store andget the app. In either case, the page or app lets her interact with anearby drone. The interaction offers something extra than merelydownloading and running an app. The app could be related to the outdoorevent, or to another outdoor activity. And she could get bonus points ifshe interacts with the drone.

The blimp screen could show data or information about the users whointeract with the drones. For example, a drone might take a photo of auser who interacted with it (in the manner of steps in this submission).This photo can be sent to the blimp screen, as incentive for users tointeract with the drones.

The blimp screen can show a running tally of how many users nearby haveinteracted with the drones. If this number reaches some set amount, thenall such users could get a bonus in the app or page. This acts asincentive for others watching to participate, to get what is in effect agroup discount. This feedback loop is a gamification of the interactionbetween the blimp, its drones and the users; separate from whether theapp or page in question is for a game.

FIG. 10 is a flow chart of a blimp and drone interacting with users onthe ground. Item 10 a 1 is the blimp showing a message on its screen.The message can tell the user how to get a particular app from an appstore. Item 10 a 2 is the user getting the app and running it on herphone. Item 10 a 3 is the user running the app and interacting with thedrone. The app could have an option to summon the drone, and the appgives the game server its device location. The game server then forwardsthis to the drone. (Perhaps via a blimp server or drone server.)

Item 10 a 4 is the drone taking a photo of the user and her phonescreen. Here, the app may have made a unique image that is shown on thephone screen. Item 10 a 5 is the drone sending that image to the blimp.This can be via the drone sending it to the drone server, which sends itto the game server, which sends it to the blimp. Item 10 a 6 is theblimp showing the user's image on the blimp screen. Along with possiblya running total of how many people have installed and run the app.

Variants of FIG. 10 and the last 2 paragraphs are possible. For example,the paragraphs and FIG. 10 assume that the user installs the app viainformation from the blimp screen, so that when the drone approaches theuser, she already has the app on her device. Clearly, one variant usesthe steps in section 1, where the user puts her electronic address (likeher phone number) on her screen and she raises her screen. A droneapproaches and scans and gets her address. It tells the game server,which sends a message to her address with, for example, a linket or deeplink for the app.

Separate from this is another way the blimp can be used. It can have acamera that shows images of the crowd. The images can be used to directdrones to cover as much of the crowd as possible. Plus, suppose that thedrones have been used for some time and users have interacted anddownloaded and are running an app publicised by the blimp and drones.

Suppose the app gets the location of the mobile device that it runs on.The crowd images from the blimp can be combined with the distribution ofthe app users for further guidance for the drones. A drone could bedirected to a portion of the crowd that has relatively few instances ofthe app. The directions to the drone can be done by the drone server ifit gets the crowd images from the blimp. Or the blimp can have a serverdoing this, if that server gets the app locations from the app server.And if the blimp server can direct the drones (which it likely can).

The steps of the previous 2 paragraphs are independent of whether theblimp has an electronic billboard or whether that billboard is used assuggested earlier in this section.

Instead of or in addition to using the billboard, the blimp could useother methods to inform the crowd. It might have a Bluetooth transmitterthat transmits a signal. A person on the ground decodes it. The signaltells how to interact with the drones. Or the blimp could have a WiFihotspot. Which may or may not connect to the Internet. In either case aperson connecting to the hotspot via her phone might see instructions onaccessing the drone.

14] Billboard;

The previous section discussed in part the use of a blimp having anelectronic billboard. This section describes the use of a stationaryelectronic billboard. The billboard can be in a permanent fixedlocation, like by a road, on or on top of a building. It can be in atemporary fixed location. For example, it might be on a flatbed truckdriven to a location with many pedestrians nearby. Like in front of amovie theatre, nightclub, restaurant, convention center or stadium. Thebillboard is controlled by a computer. The computer has Internet access.

There are 1 or more drones flying near the crowd. The billboard showsinstructions explaining how a pedestrian with her cellphone or othermobile device can interact with a drone. The interaction can be by thevarious means described earlier. The billboard can also show a runningtally of how many pedestrians have interacted with the drones. Perhapsjust for informational purposes. But this could also be a gamificationto induce more people who have not yet done so to interact with thedrones.

The interaction can be via a user installing and running an app. Perhapsvia getting and using a deep link or linket from a drone or equivalentlyfrom a drone controller.

The drone controller can be different or be the same as the billboardcontroller. The billboard controller might control some drones nearby,while other drones nearby are under the separate control of dronecontrollers. This may be for scenarios where in large crowds, the dronesof the billboard controller are inadequate to meet the demands of thecrowd. So the billboard controller calls in (i.e. hires) other drones.

The gamification could be where users who interact with the drones andget the app are encouraged to recruit others nearby. Perhaps if enoughdownload and use the app, all in the vicinity of the billboard get extracredits or compensation (monetary or not). A volume discount.

The recruiting by someone, Sheila, who interacted with a drone does notnecessarily require that her recruited friend, Tim, interact with adrone. For example, suppose Tim's friend Sheila is nearby and she didinteract. When she runs the app, it can encourage her to sendinstallation information (perhaps in the form of a deep link or linketthat points to the app in an app store) to friends. Sheila already hasTim's electronic address, so she forward him the information and he doesthe install and runs the app.

A generalisation is that Tim might not even be near Sheila and thebillboard. The app can decide whether to require that Tim be nearby ornot. If there is such a requirement, the app can (try to) enforce thisby using the location information from Tim's phone. Tim's location isnot verified by the drone. This might not matter to the app firm. Theymight just want more users.

If the billboard shows a running tally of how many users have interactedwith the drones, this could include users near other billboards. Theremight be a promotional campaign across several outdoor locations in acity, or in different cities, at the same time. One reason for the tallyto be computed this way is to get large numbers shown on the billboard.To encourage more group participation.

The use of multiple billboards can include a billboard on a blimp, wherethe latter was described in the previous section. Or billoards onseveral blimps.

Digital signage in shop windows can also be used in addition to or inplace of billboards.

We claim:
 1. A method of a drone controlled by a ground-based controllerdevice; the drone has a camera, geolocation and knows an orientation ofthe camera; the drone is airborne and near a person on the ground; theperson being different from a human operating the controller; the dronereceives a wireless signal from a mobile device of the person; thesignal is decoded to an electronic address of the person or mobiledevice; the controller sends a message to the address; the controllerperforms a wireless interaction with the mobile device.
 2. The method ofclaim 1, where the drone uses the camera to scan a screen of the mobiledevice; the signal is decoded from a scanned image.
 3. The method ofclaim 2, where the scanned image is a barcode.
 4. The method of claim 3,where the barcode is a QR code.
 5. The method of claim 3, where thebarcode is a Data Matrix code.
 6. The method of claim 2, where thescanned image depicts Unicode characters; where Optical CharacterRecognition is used to decode the image.
 7. The method of claim 1, wherethe drone decodes an audio signal from the mobile device.
 8. The methodof claim 1, where the drone decodes a combination of an audio signal anda visual signal from the mobile device.
 9. The method of claim 1, wherethe drone uses image recognition to detect a posture of the person; thedrone maneuvering close to the person, to image a screen of the mobiledevice.
 10. The method of claim 9, where the posture includes anupraised hand of the person; the hand holding the mobile device.
 11. Themethod of claim 1, where the signal is a phone number; the controllerringing the number; the controller having an automated assistant programinteracting with the mobile device.
 12. The method of claim 1, where thesignal is a phone number; the controller sending a text message to thenumber.
 13. The method of claim 1, where the signal is a deep link; thedeep link having an id of an app in an app store; the controller usingthe app to interact with the mobile device.
 14. The method of claim 13,the deep link having an identifier of an Internet address of the mobiledevice; the controller using the app to send messages to the address.15. The method of claim 1, where the signal is a linket; the linket is abit string; the controller presenting the linket to a Registry server;the controller receiving a deep link from the Registry; the deep linkhaving an id of an app in an app store; the controller using the app tointeract with the mobile device.
 16. The method of claim 1, where thedrone sends images or video to the mobile device.
 17. The method ofclaim 16, where the controller cedes partial control of the drone to themobile device for a given amount of time.
 18. The method of claim 16,where the drone has audio input from a microphone embedded in the drone;the drone sending audio data to the mobile device.
 19. The method ofclaim 1, where the message sent by the controller includes an URL of awebpage served by the controller; the webpage lets the mobile devicerent the drone.
 20. The method of claim 1, where the message sent by thecontroller includes a deep link; the deep link has an id of an app in anapp store; the controller interacts with an instance of the app on themobile device.